Thai Word Segmentation with Hidden Markov Model and Decision Tree
Identifieur interne : 000821 ( Main/Exploration ); précédent : 000820; suivant : 000822Thai Word Segmentation with Hidden Markov Model and Decision Tree
Auteurs : Poramin Bheganan [Australie] ; Richi Nayak [Australie] ; Yue Xu [Australie]Source :
- Lecture Notes in Computer Science [ 0302-9743 ] ; 2009.
Abstract
Abstract: The Thai written language is one of the languages that does not have word boundaries. In order to discover the meaning of the document, all texts must be separated into syllables, words, sentences, and paragraphs. This paper develops a novel method to segment the Thai text by combining a non-dictionary based technique with a dictionary-based technique. This method first applies the Thai language grammar rules to the text for identifying syllables. The hidden Markov model is then used for merging possible syllables into words. The identified words are verified with a lexical dictionary and a decision tree is employed to discover the words unidentified by the lexical dictionary. Documents used in the litigation process of Thai court proceedings have been used in experiments. The results which are segmented words, obtained by the proposed method outperform the results obtained by other existing methods.
Url:
DOI: 10.1007/978-3-642-01307-2_10
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 001D26
- to stream Istex, to step Curation: 001C06
- to stream Istex, to step Checkpoint: 000343
- to stream Main, to step Merge: 000829
- to stream Main, to step Curation: 000821
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct:series"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Thai Word Segmentation with Hidden Markov Model and Decision Tree</title><author><name sortKey="Bheganan, Poramin" sort="Bheganan, Poramin" uniqKey="Bheganan P" first="Poramin" last="Bheganan">Poramin Bheganan</name></author><author><name sortKey="Nayak, Richi" sort="Nayak, Richi" uniqKey="Nayak R" first="Richi" last="Nayak">Richi Nayak</name></author><author><name sortKey="Xu, Yue" sort="Xu, Yue" uniqKey="Xu Y" first="Yue" last="Xu">Yue Xu</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:4A4D9B448F63BBBA05EFF6F79110AE079A59D898</idno><date when="2009" year="2009">2009</date><idno type="doi">10.1007/978-3-642-01307-2_10</idno><idno type="url">https://api.istex.fr/document/4A4D9B448F63BBBA05EFF6F79110AE079A59D898/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">001D26</idno><idno type="wicri:Area/Istex/Curation">001C06</idno><idno type="wicri:Area/Istex/Checkpoint">000343</idno><idno type="wicri:doubleKey">0302-9743:2009:Bheganan P:thai:word:segmentation</idno><idno type="wicri:Area/Main/Merge">000829</idno><idno type="wicri:Area/Main/Curation">000821</idno><idno type="wicri:Area/Main/Exploration">000821</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Thai Word Segmentation with Hidden Markov Model and Decision Tree</title><author><name sortKey="Bheganan, Poramin" sort="Bheganan, Poramin" uniqKey="Bheganan P" first="Poramin" last="Bheganan">Poramin Bheganan</name><affiliation wicri:level="1"><country xml:lang="fr">Australie</country><wicri:regionArea>Faculty of Science and Technology, Queensland University of Technology</wicri:regionArea><wicri:noRegion>Queensland University of Technology</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Australie</country></affiliation></author><author><name sortKey="Nayak, Richi" sort="Nayak, Richi" uniqKey="Nayak R" first="Richi" last="Nayak">Richi Nayak</name><affiliation wicri:level="1"><country xml:lang="fr">Australie</country><wicri:regionArea>Faculty of Science and Technology, Queensland University of Technology</wicri:regionArea><wicri:noRegion>Queensland University of Technology</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Australie</country></affiliation></author><author><name sortKey="Xu, Yue" sort="Xu, Yue" uniqKey="Xu Y" first="Yue" last="Xu">Yue Xu</name><affiliation wicri:level="1"><country xml:lang="fr">Australie</country><wicri:regionArea>Faculty of Science and Technology, Queensland University of Technology</wicri:regionArea><wicri:noRegion>Queensland University of Technology</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Australie</country></affiliation></author></analytic><monogr></monogr><series><title level="s">Lecture Notes in Computer Science</title><imprint><date>2009</date></imprint><idno type="ISSN">0302-9743</idno><idno type="eISSN">1611-3349</idno><idno type="ISSN">0302-9743</idno></series><idno type="istex">4A4D9B448F63BBBA05EFF6F79110AE079A59D898</idno><idno type="DOI">10.1007/978-3-642-01307-2_10</idno><idno type="ChapterID">10</idno><idno type="ChapterID">Chap10</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">0302-9743</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Abstract: The Thai written language is one of the languages that does not have word boundaries. In order to discover the meaning of the document, all texts must be separated into syllables, words, sentences, and paragraphs. This paper develops a novel method to segment the Thai text by combining a non-dictionary based technique with a dictionary-based technique. This method first applies the Thai language grammar rules to the text for identifying syllables. The hidden Markov model is then used for merging possible syllables into words. The identified words are verified with a lexical dictionary and a decision tree is employed to discover the words unidentified by the lexical dictionary. Documents used in the litigation process of Thai court proceedings have been used in experiments. The results which are segmented words, obtained by the proposed method outperform the results obtained by other existing methods.</div></front></TEI><affiliations><list><country><li>Australie</li></country></list><tree><country name="Australie"><noRegion><name sortKey="Bheganan, Poramin" sort="Bheganan, Poramin" uniqKey="Bheganan P" first="Poramin" last="Bheganan">Poramin Bheganan</name></noRegion><name sortKey="Bheganan, Poramin" sort="Bheganan, Poramin" uniqKey="Bheganan P" first="Poramin" last="Bheganan">Poramin Bheganan</name><name sortKey="Nayak, Richi" sort="Nayak, Richi" uniqKey="Nayak R" first="Richi" last="Nayak">Richi Nayak</name><name sortKey="Nayak, Richi" sort="Nayak, Richi" uniqKey="Nayak R" first="Richi" last="Nayak">Richi Nayak</name><name sortKey="Xu, Yue" sort="Xu, Yue" uniqKey="Xu Y" first="Yue" last="Xu">Yue Xu</name><name sortKey="Xu, Yue" sort="Xu, Yue" uniqKey="Xu Y" first="Yue" last="Xu">Yue Xu</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000821 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000821 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:4A4D9B448F63BBBA05EFF6F79110AE079A59D898
|texte= Thai Word Segmentation with Hidden Markov Model and Decision Tree
}}
| This area was generated with Dilib version V0.6.32. |  |